# GPTQ Quantization

This is the result of GPTQ quantization on Qwen/Qwen3-Reranker-4B, significantly reducing VRAM usage.

GPTQ quantized version of Qwen3-Embedding-0.6B, optimized for video memory usage with minimal performance loss

Qwen3-0.6B is the latest 0.6B-parameter large language model in the Qwen series, supporting switching between thinking and non-thinking modes, with exceptional reasoning, instruction-following, and agent capabilities.

Qwen3 is the latest version in the Tongyi Qianwen series of large language models, offering a 1.7B-parameter GPTQ 8-bit quantized model that supports switching between reasoning and non-reasoning modes, enhancing inference capabilities and multilingual support.

Orpheus-3B-0.1-FT is a text-to-speech model based on a causal language model, supporting efficient quantization compression.

This is the GPTQ-Int4 quantized version of the Qwen2.5-VL-3B-Instruct model, suitable for multimodal tasks involving image-to-text and text-to-text, supporting both Chinese and English.

This is the INT4 quantized version of the Meta-Llama-3.1-8B-Instruct model, quantized using the GPTQ algorithm, suitable for multilingual dialogue scenarios.

A QWEN-based AI TCM consultation model integrating Five Elements theory to provide personalized TCM diagnostic services

GPTQ quantized version of Meta's Llama 2 13B model, suitable for efficient inference